In the history of film production, the cost of the film and its development determined the frame rate. In the beginning, a variety of different frame rates were used. Eventually, 24 frames per second (fps) became the standard film frame rate. Today, even though digital cameras can shoot at a variety of frames per second, 24 fps is usually chosen. This results in audiences being comfortable with the feel of most moving images at this frame rate. Viewers see higher frame rates as having a cheaper video, or less cinematic feel to them.
In the dark cinema environment, the projector projects the films to the screen by using a double (48 fps) or in most cases triple (72 fps) flashing of the content. It takes a finite amount of time to move the film from one frame to the next and during that time the shutter needs to close so that the movement is blocked from being projected. The eye can notice the flicker at frame rates below this. While this works for still images, the overall frame rate is still 24 fps. Therefore high contrast or high luminance edges will have a strobing artifact. Because people are accustomed to the 24 fps film performance, small amounts of strobing are usually not noticed and found acceptable. Directors and cinematographers try to minimize objectionable strobing by controlling lighting, movement and the shutter angle of the camera.
High dynamic range technology, as used here, means technology that have a difference between the blacks and whites of 5 orders of magnitude. HDR has existed for several years, but only recently have displays been able to display HDR. HDR may result in more and more content that contains objectionable amounts of judder because the peak luminance and contrast of the content has been increased. So while the HDR film will bring more video fidelity, color and details; it also makes the 24 Hz strobing worse.
To reduce the 24 Hz strobing effect, some approaches use frame rate conversion methods to generate high frame rate content for HDR display. In addition, some directors have shot and released films shot at higher frame rate such as 48 and 120 fps. Audiences have not universally accepted these as an improvement. What is needed is a method to maintain the look and feel of 24 fps content while reducing the strobing to an acceptable level.
When films are shoot at 120 fps and then played in a traditional 24 Hz cinema, typically the playback drops most of the original frames causing a too-small shutter angle that results in an unnatural feel to because of more noticeable strobing artifacts and a more noticeable stutter to the motion. While combining original frames together can increase the shutter angle, the adjustment is coarse and limits the original content to being shot at a 360 degree shutter angle. For example, if the original was shot at 120 fps and a 360 shutter angle, the choices for the shutter angle at 24 fps are 72, 144, 216, 288 or 360. This may not provide the flexibility required to trade-off blur versus strobing artifacts.
The traditional way of reducing the strobing artifact increases the exposure time during filming causing any moving object to blur. In cinema, the exposure time is referred to the shutter angle. A 360 shutter angle results in an exposure equal to inverse of the frame rate. A 180 shutter angle has ½ that exposure. The longer the exposure time, the more the moving edge blurs and a lower amount of the edge flicker or strobing produced. The downside of long exposures results in portions of the scene without objectionable strobing to also blur.
Traditionally, the shutter angle or exposure time of the 24 fps video camera is set to ½ of the frame rate, equaling a shutter angle of 180 degrees, which can get a natural looking image with motion blur. With the shutter angle at 180 degrees, the exposure time of each frame in the 24 fps film equals 1/48 second, and the exposure time of each frame in the 120 fps film equals 1/240 second. The film director can change the shutter angle for different use cases and different scenes. Larger shutter angles will create a blurrier image, and a smaller shutter angle will produce a sharper image but with more stutter.
Existing methods of converting high frame rate content to lower frame rates have been constrained to dropping or creating a weighted average of a couple of input or output frames together and/or inserting new interpolated frames at new output time intervals, or replacing existing frames with interpolated frames. US Patent Publication US2017/0054938 to Xu discloses several such methods. These existing methods do not provide the same quality or flexibility that a cinematographer has in the field with respect to both selecting the frame rate and controlling the amount of blur in the shot.